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Abstract. We present a complete mean field theory for a balanced state of a simple 
model of an orientation hypercolumn. The theory is complemented by a description 
of a numerical procedure for solving the mean-field equations quantitatively. With 
our treatment, we can determine self-consistently both the firing rates and the firing 
correlations, without being restricted to specific neuron models. Here, we solve the 
analytically derived mean-field equations numerically for integrate-and-fire neurons. 
Several known key properties of orientation selective cortical neurons emerge naturally 
from the description: Irregular firing with statistics close to - but not restricted to 
- Poisson statistics; an almost linear gain function (firing frequency as a function of 
stimulus contrast) of the neurons within the network; and a contrast-invariant tuning 
width of the neuronal firing. We find that the irregularity in firing depends sensitively 
on synaptic strengths. If Fano factors are bigger than 1, then they are so for all 
stimulus orientations that elicit firing. We also find that the tuning of the noise in the 
input current is the same as the tuning of the external input, while that for the mean 
input current depends on both the external input and the intracortical connectivity. 
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1. Introduction 



Neurons in primary visual cortex (VI) fire highly irregularly in response to visual stimuli, 
but with reproducible firing rates. They do so despite the fact that they receive synaptic 
input from thousands of other cortical neurons, which would lead to fluctuations in 
the input that were small compared to the mean if excitatory and inhibitory inputs 
were not balanced pQ. There has been some success in describing how such a balance 
can emerge self-consistently from dynamics that are plausible for cortical networks. 
This was accomplished by mean field-descriptions by van Vreeswijk and Sompolinsky 
[21 Ej and Amit and Brunei jUElEj- However, their treatments do not permit a self- 
consistent calculation of firing correlations. How to do this correctly was first shown 
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for an all-inhibitory network by Hertz et al. J.7J using the systematic formulation of 
mean field theory due to Fulvi Mari jH] . In a recent paper [5] we presented a mean-field 
theory for a balanced network model that allowed us to quantify how the irregularity in 
firing and, more generally, the firing correlations depend on intrinsic network properties 
such as synaptic strengths. The analysis was applied to a statistically homogeneous 
network, representing a cortical column composed of neurons with similar response 
characteristics. Here, we show how to extend this treatment to networks with systematic 
structure, consisting of multiple cortical columns. In particular, we model an orientation 
hypercolumn, composed of a set of orientation columns. 

An orientation column contains neurons that respond strongest to elongated visual 
stimuli of a specific orientation, the preferred orientation (PO). Orientation selective 
neurons exhibit a tuned response to other orientations, with sharply decreasing firing 
rates as the similarity between PO and stimulus orientation decreases, until the firing 
is completely suppressed for orientations outside the tuning width of the neuron in 
question. An important feature of orientation tuning is that the tuning width is 
independent of the stimulus contrast [TU]. It is not possible to capture this feature 
in a single-neuron description using a Hubel and Wiesel feed-forward connectivity ^T] 
from the lateral geniculate nucleus (LGN); rather, cortical interactions are needed to 
achieve contrast-invariant tuning (for review see 12J). Ben-Yishai et al [13J proposed a 
model for which the tuning width is independent of the contrast, but a threshold-linear 
relationship between input current and firing rate was an assumption of the model, and 
the problem of the firing statistics was not addressed. 

Here, we show how a contrast-invariant tuning width, an almost linear input- 
output relationship, and irregular firing can all be explained by a balanced hypercolumn 
model. With our mean- field treatment, we can quantify how certain network properties 
like synaptic strengths, tuning of the LGN input and of the intracortical connectivity 
influence the statistics and tuning of the neuronal firing. Using the Fano factor F (the 
ratio of spike count variance and mean spike count) to quantify the irregularity in firing, 
we find, e.g., that if F is significantly greater than 1 the orientation tuning of F reaches 
a maximum at the PO (Fano factors greater than 1 are normally observed for neurons in 
VI [14J). We also make quantitative predictions about the tuning of the input currents 
and their fluctuations. 

2. Model and Methods 

We model a single orientation hypercolumn in primary visual cortex, with a simplified 
network architecture as indicated in Figure H The network comprises an excitatory 
population and an inhibitory one, of sizes Ni and N2, respectively. Each population is 
divided into n sub-populations (orientation columns), parameterized by an angle 9. The 
angles, spaced equally between — n/2 and n/2, indicate the preferred orientation (PO), 
to which the neurons in the corresponding column respond strongest. 

We use leaky integrate-and-fire neurons and interconnect them randomly with a 
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Figure 1. Structure of the model network. The hypercolumn consists of multiple 
orientation columns, each of which has an excitatory and an inhibitory subpopulation 
and is assigned a preferred orientation (PO) 9. Columns with more similar POs share 
on average more connections than more dissimilar ones (the density of connections is 
indicated only between one column and the rest, for clarity). The network receives 
excitatory external input, weakly tuned to the stimulus orientation 8q. The inset 
shows a sketch for the connectivity and connection strengths J a t within an orientation 
column. 



connection probability P a b{9 — 9') that depends on the similarity of the POs. The 
probability that a neuron with PO 9 (in population a) receives afferent input from a 
neuron with PO 9' in population b is taken as 

P ab (9 -9') = ^(l + 1 cos 2(9-9')), (1) 

where Kb is the expected overall number of inputs from neurons in population b. We take 
the ratio K^/N^ independent of b, i.e., excitatory and inhibitory neurons interconnect 
with the same probability in our model. The functional form of (JIJ is motivated by 
anatomical evidence that the connection probability between cortical neurons decreases 
as their distance increases, and by the fact that orientation columns with similar PO 
tend to lie closer together on the cortical surface than ones with dissimilar PO. We 
followed Ben-Yishai et al. |13] in choosing the simplest possible form that is periodic 
with period 7r. We assume that the degree of tuning, as measured by the parameter 
7 G (0, 1), is the same for both the inhibitory and the excitatory population. 

Each nonzero synapse from a neuron in population b to one in population a is taken 
to have strength 



T a.e,b0' _ Jab (r) \ 



where the parameters Jab are of order 1. With this scaling, the fluctuations in the input 
current are also of order 1, the same order as the distance between reset and threshold 
of our model neurons (cf. van Vreeswijk and Sompolinsky [21 El)- 
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The subthreshold dynamics of the membrane potentials are given by 

^> = + i:f(e ) + if ' rec (t), (3) 

where the membrane time constant r is chosen to be the same for all neurons. The 
excitatory external input from the LGN, 1^(60), is assumed to be (weakly) tuned to 
the orientation 8q of the stimulus due to a feed-forward connectivity from the LGN as in 
the classical model by Hubel and Wiesel [TT]. For simplicity, we take it to be constant 
in time and the same for all neurons i within a column. The functional form we use is, 
similar to the tuning of the intracortical connectivity, 

l£(0 o ) = I?'(l + e<x* 2(6-60)), (4) 

where e G (0, 1) is the degree of tuning, which is assumed to be the same for both 
populations. (The condition e < 1 assures /^(^o) to be non-negative, i.e. excitatory, 
for all orientations). A more detailed model for this external input current, including 
temporal fluctuations and random connectivity, was briefly described in an overview 
article by Hertz et al. [TK] . 

The recurrent input 7° ,rec (t) from within the model cortex is given by 

2 n N b /n 

if' Tec (t) = J2 E E Jg^sf®, (5) 

b=ie'=e 1 j=i 

where Sj 9 ' (t) = J2 S $(t — tje'b) 1S t ne spike train of neuron j with PO 6' in population b. 



Mean Field Theory 

In the following mean-field description of the orientation hypercolumn model, we 
consider stationary firing only, for simplicity. However, the formulation is general enough 
to allow for non-stationary rates. We presented such a time-dependent treatment for a 
balanced single-column model elsewhere 0. 

Because of the dilute random connectivity, each neuron receives a high number of 
uncorrelated inputs (we assume Ifj, to be large, but smaller than JVj,). According to 
the central limit theorem, the recurrent input currents given by (0) can therefore be 
described as Gaussian random processes. For stationary rates, the mean input current 
is constant in time for any given neuron, although the level of the mean does vary from 
neuron to neuron due to the random connectivity. In a general mean-field theory, one 
must consider temporal correlations in these currents, i.e., not restrict the description 
of the random processes to white noise. 

To separate the mean of the currents from their fluctuations ("noise"), it is 
convenient to apply such separations to the description of both the synaptic weights 
J°"j' be and the spike trains Sj 9 '(t) in (j5J). For the weights we can write 

jad,be' T ae,b6' . r T a6,be' / a \ 

Jij =Jij + 6J ij > (6) 
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where the bar means averaging over the index j, i.e., the neurons in the source 
population: 

1 N b /n 

jaW> = J_ V jf W (7) 

13 N b /n » v ' 

Generally, we use the bar-notation for averaging over neuron populations, which will 
always apply to the running index j in this work. To separate the spike trains into 
static and dynamic components, we write 

Sf(t)=r b (9') + 5rf + 5Sf(t), (8) 

where r b {9') = r h ®' = l/(N b /ri)Y,j r f is the average rate of the neurons in sub- 
population 9' of population b. The difference between this average rate and the actual 
rate of neuron j is denoted b~r b ® ' ■ These two components are both static, describing 
time-averaged quantities. The temporal fluctuations of the spike train and their possible 
correlations in time are captured by the third term on the right-hand side of (JHJ), 5Sj 6 ' (t). 
Using the central limit theorem and methods like those in [H] and jTH] we can then derive 
the following mean- field formulation of the recurrent current: 

IlTit) = E Jab (V^A + \A - K b /N b B b (t)) , (9) 
6=1 ^ ^ 



with 



A b =-Y t (i + 1 cos2(e-e'))r b (e') (10) 



71 6»'=6»i 



1 8n / 1 \ 

B b (t) = - E + 7 cos 2(9 - 9') ( ((rf f) 1 x w + &(t) (11) 



71 6'=6 X 

where the values x b Q> are drawn from a unit-variance normal distribution. Selecting 
specific values xw effectively samples different neurons within the column population. 
We have dropped the neuron index i because this statistical description of the input 
current reduces the network problem to single neuron problems - one for each column 
population, indexed by a9. The terms £,w>(t) stand for realizations of Gaussian random 
processes obeying 

(Hw>m w (t')) = c w (t-t'). (12) 

Here, Cw(t — t') denotes the average autocorrelation function of the fluctuations in the 
spike trains of neurons with PO 9' in population b, given by 

i N b /n 

Cw{t - t>) = — - £ (5Sf(t)5Sf(t')). (13) 

With the operation (•) we mean averaging over realizations of random processes, such as 
stochastic spike trains. We will refer to such realizations as "trials" since they represent 
(responses to) repeated presentations of the same stimulus in experimental settings. 
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The balance condition 

The input currents from the excitatory population and the inhibitory population have 
mean values of order \fK[ 3> 1 and ^/Kq, 3> 1? respectively (see Equation ©). In 
addition, for the external input current (@J) we take J^ xt = \J KqI^ with \[K~q ^> 1. If the 
neurons are to exhibit irregular firing at a low rate, as cortical neurons do, these currents 
must nearly cancel and threshold crossings have to be caused by the fluctuations in the 
currents, which are of order 1. For our orientation hypercolumn model, this balance 
condition implies that the average input currents in (jSJ) have to add up to zero for each 
orientation column 9: 

2 

sJK i^{l + e cos 2(0 - O )) + ]T J ab ^jK b A b = 0(1), (14) 

6=1 

where A b is defined in (fTUI) . Here, we have ignored the contribution of the leakage 
current (the first term on the right-hand side of (JHJ)), because it is small compared to 
the input currents, and because the balance condition (JT3j) holds only up to corrections 
of 0(1). 

To solve these equations, we consider a continuum formulation for the weighted 
average over all angles instead of the discrete formulation in ()1U|) and write 

A b = f /2 — (l + 7 cos2(0-0')W)- (15) 

J-tt/2 7T 

Then (pPfj) becomes a pair of integral equations for r a (8). 

In the broadly tuned case (all orientation columns respond with non-vanishing mean 
rates to every stimulus orientation), these integral equations can be solved directly. To 
do so, we perform a Fourier expansion centered at 9q of the mean rate within orientation 
column 6' and write r b {6') = r b0 + cos 2(0' — O ) + ■ • •. For both the input current 
and the connection probabilities, we have already used such Fourier notations with the 
fewest possible terms to retain a periodic function with period tt. Due to that choice, 
all higher Fourier components for the mean currents vanish as well, and we get 

/ — - 2 / — 1 

^K i: xt (l + ecos2(fl - )) + £ \j K b J ab [r bfi + - 7 r 6|2 cos2(0 - Q )\ = 0. (16) 

6=1 1 

By solving for each of the two Fourier components of the mean rates separately, we 
obtain 

- E^Wr (17) 



'a,0 



V2= --E(J- 1 )a6/r = -^0, 

Th 7 



where the matrix J is composed of the elements J ab = J ab ^j K b j 'K . Firing rates have to 
be non-negative, so this solution can only be valid for e G (0,7/2]. However, such 
a broad tuning is not normally observed for cortical neurons. Rather, orientation 
sensitive neurons tend to be more "narrowly tuned", with firing suppressed for stimulus 
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orientations 6q that differ too much from the neuron's preferred orientation 0: r a = 
for |0 — O | > 9 C for some tuning width C . Within the parameter regime e G (7/2,7] 
we find such narrowly tuned solutions to our model. The tuning width 9 C turns out to 
be the same for both excitatory and inhibitory neurons, which is a consequence of the 
population-independence of the tuning parameters e and 7. 

To find the solutions for the narrowly tuned case, we use our insight from the 
broadly tuned case and make the ansatz 

M J r fe . o + r bi2 cos2(0'-0 o ) for |0' - O | < C 6 
r ^ ) = ^0 for|0'-0 o i>0 c 6 , (19) 

where 9 b c = — 1/2 cos~ 1 (r b fi/r b 2 ). As mentioned above, since we have assumed equal 
tuning in (£[]), 9 b c is the same for both b. Thus, in (I15j) the integration is restricted to 
\9' — O I < @c- Because r b {9') = at 0' — O = C , we can rewrite the part of the ansatz 
for 1 0' — 0o I < C in the form 

n {9') = r 6i2 (cos2(0 / - O ) - cos20 c ). (20) 

With this approach, we can indeed find solutions for the tuning width and the rates 
from the balance condition (JT3J). Analogous to the solution for the broadly tuned case 
(|16|). now the total mean-input current can be expressed as 

(J ae ' tot ) = sjK~Jf{l + e cos 2(0 - O )) 

+ J2 \fK b J ab [r b>2 f (9 c ) + ir bt2 f 2 (9 c ) cos2(0 - O )], 



(21) 



=1 



where 



f (0 c ) = / — (cos 20' - cos 20 c ) = - (sin 20 c - 20 c cos 20 c ) (22) 

J-8 C 7T 7T 

f 6 c d9' 1 1 

/ 2 (0 C ) = / — cos20'(cos20 / - cos20 c ) = -(0 C - - sin40 c ). (23) 

(We have borrowed the notation from Ben-Yishai et al. ^3] who studied a different kind 
of model that contains similar expressions; see also [17J). Again, the total current (|21j) 
has to vanish for all orientation columns 0, so both the constant and the cos 2(0 — 0o) 
terms vanish separately: 

IT' + jlJabr^Wc) =0 (24) 
6=1 

eiT t + ljZJ ab r b , 2 f 2 {9 c ) = Q (25) 



6=1 



Dividing (j^U by yields 

h{e c ) 6 



(26) 



fo(0c) 1 

which can be solved for C . Note that (J26j) . and thus the tuning width of the mean rates, 
does not depend on the overall strength of the input, J^ xt (i.e., the "contrast" of the 
stimulus). We find therefore contrast-invariant tuning of the mean rates as a result of 
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cortical interactions, in agreement with experimental findings ^U]- Having calculated 
6 C , we can find the mean rates with help of (12411 . via 



^-tItBJ" 1 )^, (27) 

J0{t>c) b=1 

and by using the equality r a = — r a ,2 cos26 l c . 

The above calculations show how cortical interactions are responsible for a 
narrowing of the tuning of the population firing rates, relative to the tuning of the 
input to the network. We can proceed one step further in our analytical treatment of 
the mean-field model and consider the tuning of the neuronal input noise spectrum. We 
can write the dynamic noise in the input current as 

W(W(*')>=E^ / —(l + lcos2(6-6>))C bei (t-t>), (28) 

6=1 ■ / -V2 7T 

where we have used the continuum notation for the weighted averages. The correlation 
function C b o>(t — if) has a piece proportional to r b {6)5{t — t'), which gives 

Jim(|W- c M| 2 ) = £^ / , — (l + 7cos2(0-0'))r 6 (0') (29) 

6=1 J—tt/2 71 

= j2 j ab[n,2fo(0c)+^ 2 f 2 (e c )cos2(e-6 )]. (30) 
6=1 

To obtain (J30)) . we performed calculations analogous to the ones for solving the integrals 
for the rate equations. Using (}2l)j) and (J2Zj), we can then write the flat contribution to 
the noise spectrum as 

2 2 

Jim (\5Q c (u)\ 2 ) = -/r [1 + ecos2(0 - 6 )} £ J 2 ab ^J" 1 )^, (31) 

6=1 c=l 

This result states that the high-frequency limit of the neuronal input noise has the same 
orientation tuning as the external input to the neuron. 

For t ^ t', it is not possible to calculate analytically solutions to (J28|) because the 
correlation function C b g'(t — t') needs to be evaluated numerically. Similarly, the tuning 
of the irregularity in the neuronal firing (as described by, e.g., the Fano factor) can only 
be determined by solving the full mean-field model numerically. 

3. Numerical procedure 

In our simulations, we modeled the orientation hypercolumn as an assembly of 30 
orientation columns, with their preferred orientations 9 equally spaced between — 7r/2 
and 7r/2 (or between —90 and 90 degrees, as in the figure captions). We used parameter 
values corresponding to iV\ = 8000 excitatory and N 2 = 2000 inhibitory neurons, and 
a membrane time constant of r = 10 ms for all neurons. The generic intra-cortical 
connection strengths J ab in (j2J) were taken as 

/ J\\ J\2 i I 0.5 — 2 \ 



</2i J22 ) V 1 —2 



(32) 
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The synaptic strengths of the afferent inputs from the LGN are taken to be stronger 
for the excitatory neurons; specifically, in (@J), we chose If** = |/f xt . To study the role 
of the overall strength of synapses, we multiply the generic synaptic weights (including 
the strength of the external input) by a common scaling factor J s . 

We use an iterative approach that was originally developed for spin glass models 
[T5] to find self-consistent solutions of the firing statistics given by the rates r a (6), the 
rate fluctuations (r^ 9 ) 2 , and the correlations C a g(t—t'). We start with initial estimates of 
these quantities, which we obtain by using a white-noise approximation in the analytical 
treatment described above. We then generate many realizations of Gaussian synaptic 
currents using (j^J) and 0, which we use to drive single integrate-and-fire neurons. 
By collecting their firing statistics, we obtain improved estimates of the rates, rate 
fluctuations, and correlations. These are then used to repeat the cycle until the input 
and output statistics are consistent. 

For the hypercolumn model, we need to determine these firing statistics for each 
population a (excitatory and inhibitory) within each orientation column 9. However, 
because of the inherent symmetry in the network topology, at each iteration step we 
only need to run simulations for half of the columns and mirror the results to obtain 
improved statistics for the entire network. To collect the firing statistics from the 
column population ad, we run many trials of single neurons that are sampled from the 
entire column population. This is achieved by generating Gaussian input currents that 
fluctuate not only in time (by generating realizations of the dynamic and appropriately 
colored input noise in (JUJ)), but also differ in their overall size due to the random 

numbers xw in (jllj) . which reflects the fact that different neurons have in general 
different connectivity patterns. (Note that we have used here - as throughout the text 
- the indices a9 for referring to the "target column" , whereas W runs over all "source 
columns"). For a more detailed account on handling some of the subtleties in obtaining 
the correct statistics, see [Oj. 

Once the procedure converges, which takes tens to hundreds of iterations, depending 
on the set of parameters and the specific approach taken, one has obtained a set of self- 
consistent firing statistics, describing the population responses for a specific network 
input (stimulus contrast and stimulus orientation). Equipped with these population 
statistics we can then calculate input and firing statistics for individual neurons. To 
specify such a neuron, we select a set 

{2^:6 = 1,2; 6' = 6 u ...,6 n } (33) 

and keep it fixed over all trials to collect the statistics for that neuron. The xw represent 
the intrinsic variability across the population in the strength of synaptic input due to 
the randomness in the connectivity of the network. 

4. Results 

We concentrate first on results describing response characteristics of neurons obtained 
from their firing statistics. It is possible to compare these results directly with known 
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Figure 2. Contrast-invariant tuning width. Average over 100 neurons (upper-left 
panel) and three randomly chosen neurons. The parameter values for the stimulus 
tuning and the connectivity tuning were e = 0.5 and 7 = 0.625, respectively, resulting 
in a tuning width of 43.2 degrees according to the calculations. Contrast-invariant 
tuning is observed for both averaged and single-neuron tuning, despite the small 
distortions and asymmetries for single neurons. (See the text for further details) 



properties like contrast-invariant tuning, or with the variability in spike counts. We 
then describe results pertaining to properties of the neuronal input currents (and their 
orientation tunings) for the hypercolumn model. 

4-1. Tuning of the neuronal firing 

For the present model, we have shown analytically above that the tuning width of the 
column population rates is invariant with respect to the contrast of the stimulus (see 
Equation (|26|l). We investigated whether such contrast-invariant tuning is also observed 
for single, randomly chosen neurons. The number of afferent connections that a given 
neuron receives from neurons with another preferred orientation is a random number 
drawn from a probability distribution given by (JIJ. This will in general distort the shape 
of the neuron's tuning curve. Figure El shows the tuning curves of three randomly chosen 
neurons from the column with 6 = for three different contrasts I° xt = 0.5, 1, and 2. 
For our network with 30 orientation columns and 2 populations, the resulting realization 
of the random connectivity to a single neuron is therefore determined by a set of 60 
random numbers (see Equation ([3*3^1). To record the neuronal responses, these sets were 
held fixed, while the network was presented successively with stimuli of all orientations 
9q. Also shown in Figure El is the result of averaging over the tuning curves of n = 100 
randomly chosen neurons. While the averaged tuning is both smooth and symmetric, the 
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Figure 3. Tuning of the gain function. Upper panel: Spike count as a function 
of stimulus contrast, parameterized by the stimulus orientation. The input-output 
relationship is linear, and the slope decreases as the stimulus orientations 9q becomes 
more dissimilar to the neuron's preferred orientation 9 — (results shown for neuron 
5 in Figure [2J. Lower panel: Spike count ratios for two pairs of spike counts resulting 
from doubling the contrast. At the preferred orientation (PO) and for orientations 
not too far from the PO, doubling the contrast doubles the spike count. For more 
dissimilar stimulus orientations, the ratios decrease systematically. 



tuning curves of single neurons show small distortions and asymmetries. Additionally, 
the overall strength of the response varies from neuron to neuron. However, despite the 
somewhat irregular shapes, the contrast-invariance of the tuning width is preserved for 
single, randomly chosen neurons. The analytical treatment predicts a threshold-cosine 
shape of the tuning, while the curves shown here, including the averaged ones, show a 
rounded fall-off to zero with non-zero rates for angles just outside the tuning width. This 
"rounding artifact" appears to be due to a slow convergence of the numerical procedure 
at extremely low firing rates; the artifact is reduced when the algorithm is run for more 
iterations. 

In all our simulations, we observe an almost linear input-output relationship 
between stimulus contrast and firing rate, in agreement with experiments (see, e.g., 
Figure 1 in [12]). Figure El shows how the input-output relationship depends on the 
stimulus orientation. In the upper panel of Figure El the spike count is plotted as a 
function of the external input strength J^ xt , i.e. the contrast of the stimulus, for a 
single neuron (neuron 5 of Figure EJ). The slope changes systematically with stimulus 
orientation 9q, getting smaller as the difference between the stimulus orientation and 
the neuron's preferred orientation increases. The lower panel of Figure El shows the 
spike count ratios of two pairs of spike counts that resulted from doubling the stimulus 
contrast. In contrast to the upper panel of Figure El these curves show results of 
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Figure 4. Tuning of the Fano factors. Tuning curves, parameterized by relative 
synaptic strengths J s , are shown for the same three neurons as in Figure [5] and for an 
average over 100 neurons (upper left panel). The Fano factors F depend systematically 
on J s : stronger synapses lead to higher Fano factors. On average, F stays either above 
1 for all orientations or below 1 for all orientations. For F ~ 1, the tuning is almost 
flat, while it reaches a maximum (resp. minimum) at the preferred orientation for 
F > 1 (resp. F < 1). 



averaging spike counts over 100 neurons, in order to make the general tendency clearer. 
It can be seen that for the preferred orientation, doubling the stimulus almost perfectly 
doubles the spike count (this is also true for single neurons, as can be read off from 
Figure EJ) • This relationship also holds for stimulus orientations away from the PO, 
until about 20 degrees difference, which is about half the tuning width of these neurons. 
For larger orientation differences, the ratio decreases. It seems likely that at large 
orientation differences (near the tuning width) this reduction is due to the rounding 
artifact for very low spike rates discussed above. For intermediate orientation differences, 
say 20-35 degrees, the reason for the reduction is not evident to us. 

We characterize the irregularity in the neuronal firing by the Fano factor F. For 
a Poisson process F — 1, while F ^ 1 implies temporal correlations in the spike times: 
F > 1 indicates a tendency towards "bursty" spiking behavior, and F < 1 indicates 
more regular spike trains with narrower interspike interval (ISI) distributions. Figure |U 
shows the tuning of the Fano factor for three different overall connection strengths 
J s = 0.4,0.7, and 1.2. As in Figure the results for (the same) three individual 
neurons are shown, as well as an averaged tuning curve. It can be seen that the Fano 
factor depends systematically on the overall strength of connectivity: stronger synapses 
lead to more irregular spike counts. The averaged tuning curves reveal two further 
properties, which we observed consistently in all our simulations, performed with many 
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Figure 5. Analysis of Fano factor tuning: tuning of mean spike count and spike 
count variance for relative synaptic strengths J s = 0.4, 0.5, . . . , 0.9. For each J s , the 
variance stays either below the mean or above the mean for all orientations (upper and 
lower panels, respectively), resulting in ratios F < 1 and F > 1 for all orientations. 
The variance increases with J s - most sensitively at the preferred orientation (PO). For 
F«l, the variance and mean tuning curves are almost identical, resulting in an almost 
flat tuning of their ratio F, while for F ^ 1 the ratios reach a minimum/maximum at 
the PO. 



different sets of parameters: First, Fano factors are either less than 1 at all angles or 
greater than 1 at all angles. Second, if they are considerably greater than 1, they peak 
at the preferred orientation, falling off as the difference between stimulus orientation 
and PO increases; in the case where F stays below 1, the opposite tuning is observed, 
i.e., the Fano factor reaches a minimum at the preferred orientation. We can shed some 
light on the emergence of these two properties by looking at pairs of tuning curves for 
the spike count variance and the mean spike count and then systematically changing 
the connection strengths. We show these tuning curves for 6 different values of J s in 
Figure El It can be seen that both the mean and the variance peak at the PO, falling 
off towards increasing angle differences. Furthermore, for F w 1 at J s = 0.7, the tuning 
curves are nearly identical resulting in almost untuned Fano factors close to 1. For lower 
J s values, the variance curve stays entirely below the mean curve, while the opposite 
is true for J s values bigger than 0.7. Therefore, the ratio of the curves, which is the 
tuning curve of the Fano factor, stays either always below 1 or always above 1. The 
size of the spike count variance depends sensitively on the overall connection strengths 
J s . Apparently, this sensitivity is strongest at the PO, decreasing towards greater angle 
differences. Therefore, the Fano factor reaches its minimum for the cases with F < 1 



Mean field theory for a balanced hypercolumn model in VI 



14 



Excitatory neuron; J =0.5 Inhibitory neuron; J. = 0.5 




Figure 6. Autocorrelation tuning. Upper panels: Weak synapses with J s = 0.5. 
There is a dip to negative values for small time differences. It decreases in strength 
at greater time differences. The dip indicates a relative refractoriness to emitting a 
spike immediately after a previous one, resulting in Fano factors F < 1. Lower panels: 
Strong synapses with J s = 1.3. There is a hill of positive correlations for short intervals, 
falling off to zero for increasing time differences. The hill indicates a tendency toward 
clustered spikes, resulting in F > 1. The autocorrelations for excitatory neurons 
(left panels) and inhibitory neurons (right panels) show the same qualitative features, 
differing only in overall size. 

(respectively its maximum for F > 1) when the stimulus is at the preferred orientation. 

As already mentioned, Fano factors that deviate from 1 indicate temporal 
correlations in the spike trains. The nature of these correlations and their orientation 
dependence is summarized in Figure EH for a case with F < 1 (J s = 0.5; upper panels) 
and a case with F > 1 (J s = 1.3; lower panels) for both excitatory neurons (left panels) 
and inhibitory ones (right panels). For J s = 0.5, there is a negative dip for small 
time differences, indicating a relative refractoriness to emitting a spike immediately 
after a previous one. For stronger synapses (J s = 1.3) there is no such refractoriness. 
On the contrary, for strong synapses, we observe positive correlations for small time 
differences. For both strong and weak synapses, the correlations are strongest at the 
preferred orientation and decrease monotonically for less optimal stimulus orientations. 
The autocorrelations for excitatory and inhibitory neurons show the same qualitative 
features, differing only in their overall size. 

In Figure [7| we illustrate how the firing statistics depend on e and 7, which 
determine how strongly the input current and the intracortical connectivity are tuned 
(see equations (JH) and ((H), respectively). Fano factor tuning curves (left panels) and 
firing rate tuning curves (right panels) for three different combinations of e and 7 are 
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Figure 7. Dependence of the Fano factors on tuning parameters e and 7 at three 
different values of relative synaptic strengths J s . Fano factors and mean spike 
counts are shown for three different combinations of e (external input tuning) and 
7 (connectivity tuning) . The tuning of both the Fano factors and the mean counts are 
controlled by the ratio e/7. 



shown, parameterized by J s , the scaling factor for the synaptic strengths. As shown 
analytically above, the ratio e/7 determines the tuning width of the neuronal firing (see 
Equation (}26|) ). This is reflected by the identical firing tuning widths in the first and 
second row of Figure [TJ for both of which e/7 = 0.8, resulting in a tuning width of 
9 C = 43.2 degrees. The third row of Figure [7| shows results for the same external input 
tuning e = 0.5 as in the first row, but for a different ratio e/7 = 0.6. This results in 
8 C = 67.7 degrees and an accordingly broader tuning curve of the firing, plotted in the 
right panel of the third row. The curves for the Fano factor tuning in the left panels of 
Figure d suggest that the tuning of the firing irregularity is - just as the tuning of the 
firing itself - only dependent on the ratio e/7. (We consistently found this dependence 
in all our simulations.) 

4-2. Tuning of the neuronal input current 

Our analytical treatment of the balanced hypercolumn model reveals that the high- 
frequency neuronal input noise power has the same tuning as the external input. In 
Figure |H1 we show simulation results of the noise tuning for the same three combinations 
of e and 7 as in Figure For the panels in the first and the second row of Figure El 
e/7 = 0.8, but e = 0.5 and e = 0.25 in the upper and middle rows, respectively. 
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Figure 8. Dependence of the noise on tuning factors e and 7. External input 
and dynamic input noise versus tuning of the neuronal firing for the same three 
combinations of e and 7 as in Figured It can be seen that the tuning of the noise is 
determined by e, while the tuning of the firing rate is determined by the ratio e/7. 



While the tuning of the neuronal firing is identical for these two cases, the noise tuning 
is weaker in the middle row, reflecting the weaker tuning of the external input (left 
panels). The results presented in the third row of Figure |H1 show a case with a broader 
tuning of the response, resulting from a different ratio between e and 7, but with the 
same e = 0.5 as in the first row. For these two cases, the tunings on the input side - 
concerning external input and dynamic noise - are practically indistinguishable, while 
the tunings of the firing differ. Thus, the noise tuning is determined by e, unlike the 
response tuning, which depends on the ratio e/7. 

The balanced state for the orientation hypercolumn implies that the mean input 
currents (external and recurrent currents), which are each of 0(\fK a ) with K a ^> 1, 
cancel up to corrections of (9(1). It is not straightforward to calculate the tuning of the 
resulting net mean current, since the balance condition ()14)) does not allow inferences 
about its size. However, the solutions obtained by the numerical algorithm provide direct 
access to the net mean currents, which we depict in Figure 01 for the same combinations 
of e and 7 as for the noise tuning in Figure El It is clear from Figure that the tuning of 
the mean input, unlike the dynamic input noise tuning, is not determined by the tuning 
of of the external input. Rather, it seems to be the ratio e/7 that primarily determines 
it, as suggested by the almost identical tunings for the two cases with identical e/7. 
Since the tuning of the external input and that of the noise variance are the same, the 
left panels of Figure El also show how the tuning of the noise compares to that of the 
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Figure 9. Dependence of the mean input current on tuning factors e and 7. External 
input tuning and mean-input tuning versus tuning of the response for the same three 
combinations of e and 7 as in Figure [7| and Figure |SJ The tuning of the mean input is 
not determined by e; rather, as for the spike count tuning shown in the right panels, 
the ratio e/7 plays an important role. 



mean input current for the three combinations of e and 7. 
5. Discussion 

In this work, we presented a complete mean field theory for a balanced network with 
structural inhomogeneity, together with an algorithm that allows one to find the self- 
consistent solutions for the mean rates, their cell-to-cell fluctuations, and the correlation 
functions. We applied the theory to a simple model of an orientation hypercolumn in 
primary visual cortex, comprised of integrate-and-fire neurons. Despite the relative 
simplicity of the model, the resulting dynamics capture several key properties known 
about responses of orientation selective cortical neurons in vivo. Within this description, 
we can pinpoint how the resulting neuronal dynamics are controlled by parameters of 
the model, and quantify their influence. 

Specifically, we find contrast-invariant tuning of the neuronal firing not only for the 
population rates, as derived from the analytical treatment, but also for single, randomly 
chosen neurons. Moreover, the firing rate increases linearly with the strength of the 
input current (i.e., the contrast of the stimulus). Note that these are network effects 
originating in the dynamical balance between excitation and inhibition, not properties 
of isolated neurons. This is in agreement with experimental results, where such a linear 
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input-output relationship can only be found for cortical neurons in vivo, but not for 
single neurons in vitro. 

Another network effect that emerges naturally from the self-consistent dynamic 
balance, in combination with the static randomness in the connectivity, is the irregularity 
in the neuronal firing. We are able to describe it quantitatively through the correlation 
functions, which are determined self-consistently in the theory. Such firing-statistical 
issues cannot be addressed in "rate models" , which simply assume a particular relation 
between average input current or membrane potential and firing rate. While it is possible 
to calculate the firing variability in the mean-field treatment of Brunei [S] , it cannot be 
done in a self-consistent manner because of the assumption that the neuronal input is 
uncorrelated in time (white noise). Here we color the noise self-consistently. Poisson- 
like statistics (Fano factor F = 1) are only one possibility within a continuum of firing 
statistics that depend sensitively on the strengths of the synapses: stronger synapses 
generally lead to higher Fano factors. The underlying mechanism can be summarized as 
follows: Stronger synapses increase the probability of a spike shortly after reset, which 
leads to a higher tendency of spikes occurring in "clusters" , thereby increasing the spike 
count variance. A detailed account of this mechanism, involving the dependence of the 
membrane potential distribution on the synaptic strength can be found in j^j, where the 
analysis was carried out for a single cortical column. 

The mean field theory applied to the present model allows us to study tuning 
properties of both the neuronal firing and the neuronal input and their dependence on 
network parameters. Concerning the irregularity of firing, our results suggest that F 
stays either above 1 or below 1 for all orientations. Moreover, the modulation strength 
of F over angles increases, relative to the almost untuned case of F ~ 1, with increasing 
(resp. decreasing) overall values of F, reaching a maximum (resp. a minimum) at the 
preferred orientation. 

Concerning the tuning of the input currents, we find analytically that the high- 
frequency input noise power has the same tuning as the external input to the neuron 
(which in turn is determined by a Hubel-Wiesel feed-forward connectivity from the 
LGN). In our numerical calculations we observe a close fit between the tuning of the 
overall input noise and the one of the external input. This suggests that the tuning 
of the external input may be a good predictor for the noise tuning, and vice versa. In 
contrast, we find that the tuning of the mean input current does not reflect the one of 
the external input, but is predominantly determined by the ratio e/7 of the modulation 
strengths of the external input and the cortical interactions. 

Some of our results (the existence of a stable, asynchronous low-rate state, 
contrast-invariant orientation tuning, and the inverse relation between the sharpness 
of orientation tuning and intracortical tuning strength 7) were obtained previously by 
Wolf et al. OJ in an extension of van Vreeswijk and Sompolinsky's stochastic binary 
model j2 E] to a hypercolumn, but the treatment of a spiking neuron model and all the 
results for correlations of both input and output are new here. Also new is that we go 
beyond population statistics and make quantitative predictions about input and output 
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characteristics of individual neurons, which can be tested directly. 

Firing irregularity of neurons in primary visual cortex has been investigated 
experimentally for a long time (see, e.g., [23 ED 1221 ESI Ej)- Well studied is also 
the dependence of firing rate on the stimulus orientation [HUEI], but we are not aware 
of studies investigating the dependence of firing irregularity on the orientation. Our 
predictions concerning the tuning of the input currents (for both mean and noise) can be 
tested experimentally by systematically changing e (the external input tuning strength) 
via changing the spatial modulation of the stimulus and then observing how the the 
mean and noise tunings are affected separately. 

The mean field theory presented here, in combination with the numerical procedure 
for finding the self-consistent solutions, can be applied to models that capture more 
of the known neuronal and cortical physiology. For example, it is straightforward to 
incorporate conductance-based synapses into the hypercolumn model, as has already 
been done for a single-column model (see j2S] and [IS]). It is also straightforward to use 
different, possibly more realistic neuron models - even several kinds of neuron models 
within one given network model, since the neuronal dynamics are explicitly simulated 
within the numerical procedure for collecting the firing statistics. Here, we have shown 
how the theory can be applied to networks with non-homogenous architecture, using a 
simple one-dimensional model for a cortical hypercolumn. This model can be thought of 
as describing an annulus around a pinwheel center. Using the same general techniques as 
introduced here, the model can be extended to incorporate a two-dimensional geometry 
to describe an entire pinwheel. Similarly, as we have shown for orientation selectivity, 
it is possible to include other coding features, such as spatial phase, for example. Thus, 
the power of this mean-field approach lies in its generality, which makes it possible to 
quantify dynamics of balanced, highly connected networks. 
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